Design Sensitivity in Observational Studies

If an observational study were free of unobserved bias, we would not know this from the observed data. The best we could hope to say is that only very large unobserved biases could alter the conclusions of the study. What features of the design of an observational study reduce its sensitivity to unobserved biases? How can designs for observational studies be compared quantitatively in terms of their ability to resist unobserved biases? The talk is from [30], [31] and [39], the last joint with Dylan Small. 1. Notation and Review 1.1. Review: Notation for a Paired Randomized Experiment[11] [21] [36] [2]. Observed covariate x and an unobserved covariate u. I pairs, i = 1; : : : ; I, of two subjects, j = 1; 2, one treated, one control, matched for x, so xi1 = xi2, but not matched for u, so typically ui1 6= ui2. Zij = 1 if j received the treatment in pair i, and Zij = 0 if j received the control, so Zi1+Zi2 = 1: Subject (i; j) has two potential responses, (rTij ; rCij), rTij observed under treatment, Zij = 1, rCij observed under control, Zij = 0, so the e¤ect of the treatment is rTij rCij ; Neyman [21] and Rubin [36]. Treatments given at doses (vTij ; vCij), possibly (vTij ; vCij) = (1; 0). Write F for f(rTij ; rCij ; vTij ; vCij ;xij ; uij), i = 1; : : : ; I, j = 1; 2g and Z for the event fZi1 + Zi2 = 1; i = 1; : : : ; Ig; then F and Z are …xed by conditioning in Fisher’s [11] theory of randomization inference. Randomization in pairs ensures Pr (Zi1 = 1 j Z; F) = 1 2 , 8i, with independently in distinct pairs. Observed response is Rij = Zij rTij + (1 Zij) rCij , observed dose is Vij = Zij vTij + (1 Zij) vCij . 1.2. Review: Randomization inference for constant treatment e¤ect[18] [27]. If treatment e¤ect is constant, = rTij rCij , then Rij = rCij+Zij , and the treated-minuscontrol di¤erence isDi = (2Zi1 1) (Ri1 Ri2) = + i where i = (2Zi1 1) (rCi1 rCi2). Test H0 : = 0 by ranking jDi 0j from 1 to I; then Wilcoxon’s signed rank statistic, W 0 , is the sum of the ranks for which Di 0 > 0, where ties are assumed absent. If H0 : = 0 is true, randomization ensures that Di 0 = i is rCi1 rCi2 or rCi2 rCi1, each with probability 12 , independently in di¤erent pairs. Given Z; F , if H0 : = 0 is true, then the jDi 0j are …xed, the Di 0 are independent, Pr (Di 0 > 0) = 12 , and each Di is symmetric about 0, soW 0 is the sum of I independent random variables taking values i or 0 each with probability 12 , i = 1; : : : ; I. A con…dence interval for is obtained by inverting the test, and the Hodges-Lehmann (HL) estimate b of is (essentially) the University of Pennsylvania, Philadelphia, PA 19104-6430 USA, [email protected]. June 2007. Supported by NSF. Reprints/overheads: http://www-stat.wharton.upenn.edu/~rosenbap/.